Acute stress facilitates habitual behavior in female rats.

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

The temporal structure of goal-directed and habitual operant behavior.

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Partial reinforcement extinction and omission effects in the elimination and recovery of discriminated operant behavior.

Three experiments explored how training reinforcement schedules and context influence the elimination and recovery of human operant behavior. In Experiment 1, participants learned a discriminated operant response in Context A before the response was eliminated with extinction in Context B. They then received a final test in each context. Groups were trained with a discriminative stimulus that predicted a reinforced response on either every trial (continuous reinforcement [CRF]) or some of the trials (partial reinforcement [PRF]). Extinction was slower following PRF training (a partial reinforcement extinction effect [PREE]) and extinguished responding increased when tested in Context A ("ABA" renewal). Experiment 2 further confirmed the PREE was obtained equally whether extinction occurred in the training context (Context A) or a new context (Context B) which is consistent with trial-based accounts of the PREE. Experiment 3 used the same design as Experiment 1 to evaluate the influence of training reinforcement on response elimination with an omission contingency. Across the omission training phase in Context B, the decrease in responding occurred more slowly in the PRF-trained group in comparison to the CRF-trained group, perhaps the first demonstration of what might be termed a PRF omission effect. Again, ABA renewal was observed in Context A. Training reinforcement schedule therefore had a similar influence on response elimination with extinction and omission. Elimination and recovery of human instrumental behavior, with extinction or omission, are influenced by training reinforcement schedule and context. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Loss aversion predicts cigarette smoking status across levels of sociodemographic characteristics.

Loss aversion (LA) is a tendency to be more sensitive to potential losses relative to similar gains. Low LA is associated with increased risk for cigarette smoking and use of other substances. Previous studies of LA and smoking risk controlled for potentially confounding influences of sociodemographic characteristics associated with smoking risk. The present study replicates these earlier observations while also examining the generality of the association between low LA and smoking risk within different levels of each of the five sociodemographic risk factors for smoking (age, educational attainment, gender, income, race/ethnicity). Parallel analyses were conducted using delay discounting (DD) as a positive control; DD is a decision-making bias regarding the rate at which rewards lose value with increasing delay to receipt. Participants were recruited using standard crowdsourcing methods and completed a sociodemographics questionnaire, a hypothetical gamble task measure of LA, and a monetary choice measure of DD. Low LA was associated with increased risk of cigarette smoking after accounting for the influence of sociodemographic characteristics and DD. Similarly, high DD was associated with increased risk of cigarette smoking after accounting for the influence of sociodemographic characteristics and LA. Further analyses showed that associations of LA with smoking risk or DD with smoking risk generally although not always remained significant within varying levels of the sociodemographic characteristics of interest. These results provide support for low LA as a reliable risk factor for smoking that has generality within and across sociodemographic characteristics and closely parallels associations observed with DD and smoking risk. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Loss Aversion and Current, Former, and Never-Smoking Status.

INTRODUCTION: Loss aversion (LA) is a bias in decision-making wherein potential losses have a greater influence on choices than equivalent gains. Such a bias may protect individuals from harm. Consistent with this idea, lower LA has been measured in individuals who endorse current cigarette smoking as well as other substance use compared to controls matched on important sociodemographic variables (age, gender, and educational attainment). The goal of the present study was to systematically replicate and extend this association between LA and smoking status by comparing those meeting criteria for current-, former-, and never-smoking status. AIMS AND METHODS: In total, 984 individuals (N = 984) that endorsed current cigarette smoking (past 30-day use; n = 361), former-smoking (no past 30-day use, >100 cigarettes lifetime; n = 317), and never-smoking (no past 30-day use, <100 cigarettes lifetime; n = 306) were recruited using standard crowdsourcing methods and completed measures of LA (50-50 gambles) and delay discounting (DD) (monetary choice questionnaire), an important decision-making bias with an established relationship to cigarette-smoking status. RESULTS: Lower LA was observed in those endorsing current smoking compared to former smoking (t[952] = -9.57, Bonferroni corrected p < .0001), and never-smoking (t[952] = -3.99, Bonferroni corrected p = .0002). LA was also greater in former- compared to the never-smoking (t[952] = -5.26, Bonferroni corrected p < .0001). This pattern did not change when accounting for DD and sociodemographics. DD results replicated prior findings. CONCLUSIONS: The results support LA as a decision-making bias related to the risk of cigarette smoking and other substance use. Further research is needed to understand the causal contributions of LA and DD and their potential intersections. IMPLICATIONS: Low LA is a risk factor for cigarette smoking. This study reports higher LA among individuals that endorsed never-smoking and former-smoking status in comparison to those endorsing current cigarette smoking. LA may influence or be influenced by a change in smoking status.

Hippocampus and neocortex send and receive predictive information.

A recent paper published in Nature by Yadav and colleagues (Nature 608 (7921): 153-160, 2022) illustrates a fundamental aspect on how different brain areas participate in memory storage and retrieval. After identifying neuron activity in the hippocampus CA1 region specific to multi-modal stimuli that predicted appetitive and aversive unconditioned stimuli (conjunctive stimulus encoding), the authors showed that neurons located in the anterior cingulate cortex dynamically encode the discrete sensory features of the outcome-predictive stimuli during training, and these highly specific feature-based projections can excite or inhibit conjunctive-coding neurons in the hippocampus during retrieval.

Intersections between environmental reward availability, loss aversion, and delay discounting as potential risk factors for cigarette smoking and other substance use.

Behavioral theory suggests that density of environmental rewarding activities and biases in decision making influence risk for substance use disorder (SUD). To better understand intersections of these potential risk factors, this study examined whether environmental reward predicted smoking status or other drug use and whether such associations were independent of two decision-making biases known to predict SUD risk, namely loss aversion and delay discounting. Individuals that reported current daily cigarette smoking (n = 186; >10 cigarettes/day) and never-smoking (n = 241; <100 cigarettes lifetime) were recruited with standard crowdsourcing methods. Participants answered questions on alcohol and other drug use. Environmental reward was assessed using the Reward Probability Index (RPI), and loss aversion (LA) and delay discounting (DD) using a gamble-acceptance task and monetary choice questionnaire, respectively. Associations of RPI, LA, and DD with cigarette smoking, alcohol use, other drug use, and combinations of co-use were examined with logistic regression controlling for sociodemographic variables (educational attainment, gender, age). Low RPI (odds ratio[OR] = 0.97, p = .006), low LA (OR = 1.22, p < .001), and high DD (OR = 1.12, p = .03), were each independently associated with increased risk for cigarette smoking, as well as other substance use, and use combinations. We saw no evidence that RPI was significantly influencing associations between LA and DD with smoking status or other substance use. Finally, RPI, but not LA or DD, was significantly associated with depressed mood and sleep disturbance. These results provide new evidence on associations of RPI with smoking status and other substance use while further documenting independent associations between LA and DD and those outcomes.

Loss aversion and risk for cigarette smoking and other substance use.

BACKGROUND: Cigarette smoking is among the leading preventable causes of global morbidity and mortality. We aimed to determine whether individual differences in loss aversion, a bias in decision-making wherein losses are valued greater than gains, predicts smoking and other addiction risk. METHODS: We recruited current daily cigarette smokers (n = 181; > 10 cigarettes per day) and never-smokers (n = 237; < 100 cigarettes lifetime) from the United States using Amazon Mechanical Turk. Groups were matched on gender, educational attainment, and age. All completed items related to current cigarette smoking, alcohol use, other drug use, sleep problems, and depressed mood, and task-based measures of loss aversion and delay discounting, a decision-making bias associated with cigarette smoking. RESULTS: Smokers were less loss averse than never-smokers (F(1, 411) = 24.19, η2 = 0.02, p < .0001) even after accounting for delay discounting (F(1, 410) = 20.53, η2 = 0.02, p < .0001). Loss aversion was also a significant independent risk factor for alcohol (F(1, 410) = 21.47, η2 = 0.02, p < .0001) and other drug use (F(1, 410) = 54.12, η2 = 0.04, p < .0001), although not other behavioral-health conditions (i.e., sleep disturbance, depressed mood). Further analyses revealed that co-occurring low loss aversion and high delay discounting were independently associated with greater risk for all patterns of substance use. CONCLUSIONS: Loss aversion was associated with current cigarette smoking and other substance use patterns independent of delay discounting. Loss aversion may warrant attention as a protective factor and potential target for preventive intervention for substance use and addiction.

Relapse after incentivized choice treatment in humans: A laboratory model for studying behavior change.

Behavior change interventions that incentivize desired behavior are highly effective for improving personal health, but difficult to maintain long term. Relapse is common and examining the mechanisms that contribute to relapse in experimental settings can identify processes relevant to substance abuse treatment. We developed a laboratory task that parallels a recent operant model of relapse after incentivized choice reported in the rodent laboratory. In two experiments, undergraduate participants first learned to make an operant response (keyboard button; R1) to earn a reinforcer consisting of an image of a preferred snack food (O1). In a second phase (Phase 2), R1 was still reinforced, but a new response (R2) was introduced and reinforced with a different reinforcer (a coin; O2). In a test phase, contingent incentives for R2 were removed (extinction) and relapse of R1 was assessed. Experiment 1 found that the O2 contingency suppressed R1 during Phase 2, and R1 relapsed rapidly in the test. Neither effect was consistently related to O2 value. Experiment 2 examined whether noncontingent presentations of O1 or O2 during the test could weaken relapse. Here, we found that noncontingent reinforcers did little to reduce or slow the increase in R1 responding. The present experiments highlight a laboratory approach to studying variables that may influence relapse after incentivized treatment. We identify and discuss areas for development to address differences between the present results and prior observations from animal and clinical studies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning.

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Reinforcer predictability and stimulus salience promote discriminated habit learning.

Instrumental (operant) behavior can be goal directed, but after extended practice it can become a habit triggered by environmental stimuli. There is little information, however, about the variables that encourage habit learning, or about the development of discriminated habits that are actually triggered by specific stimuli. (Most studies of habit in animal learning have used free-operant methods.) In the present experiments, rats received training in which a lever press was reinforced only in the presence of a discrete stimulus (S) and the status of the behavior as goal-directed or habitual was determined by reinforcer devaluation tests. Experiment 1 compared lever insertion and an auditory cue (tone) in their ability to support habit learning. Despite prior speculation in the literature, the "salient" lever insertion S was not better than the tone at supporting habit, although the rats learned more rapidly to respond in its presence. Experiment 2 then examined the role of reinforcer predictability with the brief (6-s) tone S. Lever pressing during the tone was reinforced on either every trial or on 50% of trials; habit was observed only with the highly predictable (100%) relationship between S and the reinforcer. Experiment 3 replicated this effect with the tone in a modified procedure and found that lever insertion contrastingly encouraged habit regardless of reinforcer predictability. The results support an interactive role for reinforcer predictability and stimulus salience in discriminated habit learning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Correction of response error versus stimulus error in the extinction of discriminated operant learning.

Two experiments with rat subjects separated learning about the discriminative stimulus versus the operant response in the extinction of discriminated operant learning. Each was designed to separate 2 forms of error that could generate extinction learning from an error-correction perspective: Stimulus error, where the discriminative stimulus overpredicts the reinforcer in extinction, and response error, where the response is higher than what the current reinforcer supports. Stimulus error would cause correction of the Pavlovian stimulus-reinforcer association, whereas response error could cause correction of the instrumental response through adjustment of the response-reinforcer association or direct inhibition of the response. Previous research has supported a role for prediction error in instrumental extinction but has confounded these 2 potential sources of error. Using new variations of the concurrent excitor paradigm (Experiment 1) and the overexpectation paradigm (Experiment 2), the present experiments manipulated response error while controlling stimulus error. Both demonstrated that response error plays a role in instrumental extinction. When a discriminated operant response is not reinforced, response error correction may cause the animal to learn to suppress that specific response. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Effects of conditioned stimulus (CS) duration, intertrial interval, and I/T ratio on appetitive Pavlovian conditioning.

Pavlovian learning is influenced by at least 2 temporal variables: The time between the onset of the conditioned stimulus (CS) and presentation of the unconditioned stimulus (US), and the time between successive conditioning trials (the intertrial interval [ITI]). Wagner's Sometimes Opponent Process (SOP) model (e.g., 1981) provides a rich account of the effects of varying the absolute durations of CS and ITI. However, other theories have contrastingly emphasized the role of the relative durations of CS (T) and ITI (I). Three experiments with rats used an appetitive conditioning preparation to separate the two approaches. They manipulated absolute values of I and T over a factor of 6 and compared the effect of varying T and I/T by the same factor. Conditioning was indexed by the rate of foodcup entry during training, during common tests conducted later with different combinations of I and T, and with a reinforcers-to-criterion measure. Experiment 1 found that learning with a 10-s CS was superior to that with a 60-s CS when the I/T ratio was the same. There was little evidence of learning with the 60-s CS. Experiment 2 replicated that result and extended it to show that a 60-s CS still supported little conditioned responding when I/T was increased from 12 to 72. Experiment 3 then examined intermediate CS durations between 10-s and 60-s while I/T was controlled. The results support a role for absolute CS duration rather than the I/T ratio. Explanations based on SOP and behavior systems theory are explored. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Inactivation of the prelimbic cortex attenuates operant responding in both physical and behavioral contexts.

The present experiments aimed to expand our understanding of the role of the prelimbic cortex (PL) in the contextual control of instrumental behavior. Research has previously shown that the PL is involved when the "physical context," or chamber in which an instrumental behavior is trained, facilitates performance of the instrumental response (Trask, Shipman, Green, & Bouton, 2017). Recently, evidence has suggested that when a sequence of two instrumental behaviors is required to earn a reinforcing outcome, the first response (rather than the physical chamber) can be the "behavioral context" for the second response (Thrailkill, Trott, Zerr, and Bouton, 2016). Could the PL also be involved in this kind of contextual control? Here rats first learned a heterogenous behavior chain in which the first response (i.e., pressing a lever or pulling a chain) was cued by a discriminative stimulus and led to a second stimulus which cued a second response (i.e., pulling a chain or pressing a lever); the second response led to a sucrose reward. When the first and second responses were tested in isolation in the training context, pharmacological inactivation of the PL resulted in a reduction of the first response, but not the second response. When the second response was performed in the "context" of the first response (i.e., as part of the behavior chain) however, PL inactivation reduced the second response. Overall, these results support the idea that the PL is important for mediating the effects of a training context on instrumental responding, whether the context is physical or behavioral.

Unexpected food outcomes can return a habit to goal-directed action.

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Relapse of operant behavior after response elimination with an extinction or an omission contingency.

The present study compared relapse after responding was eliminated by extinction or omission training in rats. In Experiment 1, lever pressing was reinforced with food pellets in Context A and then eliminated with either extinction or omission training in Context B. The response was then tested in Contexts A and B in either the presence or absence of free food pellets delivered on a random time schedule. All rats showed higher responding when tested in Context A than Context B, and there was little evidence that omission training attenuated this ABA renewal effect. Noncontingent pellets increased responding after extinction but not after omission. However, when responding on the last day of response elimination was compared to responding during the test in the response-elimination context, there was some evidence that omission-trained rats showed a small increase in responding even when tested with free pellets. Results of Experiment 2 suggest this increase was not due to differences in the temporal distribution of pellets during elimination and the test, and that the result might be due to mere removal of the omission contingency, but any such effect is small and difficult to detect statistically. The results provide new information about factors generating relapse after omission training.

Resurgence in humans: Reducing relapse by increasing generalization between treatment and testing.

Resurgence is the increase in performance of an extinguished instrumental (operant) response that accompanies the extinction of a response that has been reinforced to replace it. Resurgence may involve processes that are relevant for understanding relapse in applied and clinical settings. While resurgence is known to be a robust phenomenon in human operant extinction, the processes that control it remain unclear. Here we asked whether human resurgence is controlled by processes that are similar to those that have been identified in animals by asking whether two methods that reduce resurgence in animals also reduce it in humans. Participants first learned to make an operant response (R1) for a tangible food reinforcer (O1). In a second phase (Phase 2), R1 was extinguished while a second response (R2) was introduced and reinforced with a virtual monetary reward (USD $0.10 coins; O2). In a test phase, extinction was then introduced for R2 and resurgence of R1 was assessed. In Experiment 1, resurgence that occurred after the treatment just described was attenuated if there had been periodic exposure to R2 extinction during the treatment phase (Phase 2). In Experiment 2, resurgence was prevented when O2, but not O1, was presented noncontingently during the test. The results are among the first to suggest a mechanism underlying resurgence in humans, namely, renewal caused by contextual change. They also provide initial evidence to suggest that resurgence may be the result of common processes in animals and humans. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior.

Goal-directed actions are instrumental behaviors whose performance depends on the organism's knowledge of the reinforcing outcome's value. In contrast, habits are instrumental behaviors that are insensitive to the outcome's current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random-interval 30-s schedule in the presence of a discriminative stimulus (S) but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the law of effect) and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Maintaining performance in searching dogs: Evidence from a rat model that training to detect a second (irrelevant) stimulus can maintain search and detection responding.

Scent-detecting dogs perform a sequence, or chain, of behaviors that, at minimum, includes searching followed by a detection behavior that signals the presence of a target stimulus to the handler. However, when working, dogs often engage in prolonged periods of searching without encountering a target. It is therefore important for trainers to use methods that promote persistent search behavior and target detection accuracy. Laboratory models can provide insights to the important variables that influence search persistence and accuracy. The present experiments examined a rat model of detection dog behavior. Two experiments assessed the use of practice with a single target stimulus to maintain search and detection of another previously-trained target. In Experiment 1, after learning a search→detection chain with two auditory targets, rats received either brief or extended training with only one of the targets before being tested for detection of both targets in extinction. The results suggest that single-target training strengthened the ability of the other target to control the detection behavior. Experiment 2 found that even infrequent target encounters were still effective at maintaining detection behavior to the other trained target. Importantly, the treatment was effective when the target stimuli were from different sensory modalities. Overall, the results support the utility of the rat model of search-dog behavior for evaluating novel training methods. We suggest several useful procedures for enhancing search persistence and accuracy in detection dogs that can be implemented in training protocols.